{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Week 3: Using RNNs to predict time series\n",
    "\n",
    "Welcome! In the previous assignment you used a vanilla deep neural network to create forecasts for generated time series. This time you will be using Tensorflow's layers for processing sequence data such as Recurrent layers or LSTMs to see how these two approaches compare.\n",
    "\n",
    "#### TIPS FOR SUCCESSFUL GRADING OF YOUR ASSIGNMENT:\n",
    "\n",
    "- All cells are frozen except for the ones where you need to submit your solutions or when explicitly mentioned you can interact with it.\n",
    "\n",
    "- You can add new cells to experiment but these will be omitted by the grader, so don't rely on newly created cells to host your solution code, use the provided places for this.\n",
    "\n",
    "- You can add the comment # grade-up-to-here in any graded cell to signal the grader that it must only evaluate up to that point. This is helpful if you want to check if you are on the right track even if you are not done with the whole assignment. Be sure to remember to delete the comment afterwards!\n",
    "\n",
    "- Avoid using global variables unless you absolutely have to. The grader tests your code in an isolated environment without running all cells from the top. As a result, global variables may be unavailable when scoring your submission. Global variables that are meant to be used will be defined in UPPERCASE.\n",
    "\n",
    "- To submit your notebook, save it and then click on the blue submit button at the beginning of the page.\n",
    "  \n",
    "Let's get started!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "deletable": false,
    "editable": false,
    "id": "BOjujz601HcS",
    "outputId": "b0a53dee-523b-4a27-b31d-2a1daed0df1c",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import unittests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating the data\n",
    "\n",
    "Let's begin by defining a bunch of helper functions to generate and plot the time series: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def plot_series(time, series, format=\"-\", start=0, end=None):\n",
    "    \"\"\"Plot the series\"\"\"\n",
    "    plt.plot(time[start:end], series[start:end], format)\n",
    "    plt.xlabel(\"Time\")\n",
    "    plt.ylabel(\"Value\")\n",
    "    plt.grid(False)\n",
    "\n",
    "def trend(time, slope=0):\n",
    "    \"\"\"A trend over time\"\"\"\n",
    "    return slope * time\n",
    "\n",
    "def seasonal_pattern(season_time):\n",
    "    \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
    "    return np.where(season_time < 0.1,\n",
    "                    np.cos(season_time * 6 * np.pi),\n",
    "                    2 / np.exp(9 * season_time))\n",
    "\n",
    "def seasonality(time, period, amplitude=1, phase=0):\n",
    "    \"\"\"Repeats the same pattern at each period\"\"\"\n",
    "    season_time = ((time + phase) % period) / period\n",
    "    return amplitude * seasonal_pattern(season_time)\n",
    "\n",
    "def noise(time, noise_level=1, seed=None):\n",
    "    \"\"\"Adds noise to the series\"\"\"\n",
    "    rnd = np.random.RandomState(seed)\n",
    "    return rnd.randn(len(time)) * noise_level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the same you have been using in the previous assignments, so you will be generating the same time series data. You can do that with the following function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def generate_time_series():\n",
    "    \"\"\" Creates timestamps and values of the time series \"\"\"\n",
    "    \n",
    "    # The time dimension or the x-coordinate of the time series\n",
    "    time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
    "\n",
    "    # Initial series is just a straight line with a y-intercept\n",
    "    y_intercept = 10\n",
    "    slope = 0.005\n",
    "    series = trend(time, slope) + y_intercept\n",
    "\n",
    "    # Adding seasonality\n",
    "    amplitude = 50\n",
    "    series += seasonality(time, period=365, amplitude=amplitude)\n",
    "\n",
    "    # Adding some noise\n",
    "    noise_level = 3\n",
    "    series += noise(time, noise_level, seed=51)\n",
    "    \n",
    "    return time, series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining some useful global variables\n",
    "\n",
    "Next, you will define some global variables that will be used throughout the assignment. Feel free to reference them in the upcoming exercises:\n",
    "\n",
    "`SPLIT_TIME`: time index to split between train and validation sets\n",
    "\n",
    "`WINDOW_SIZE`: length od the window to use for smoothing the series\n",
    "\n",
    "`BATCH_SIZE`: batch size for training the model\n",
    "\n",
    "`SHUFFLE_BUFFER_SIZE`: number of elements from the dataset used to sample for a new shuffle of the dataset. For more information about the use of this variable you can take a look at the [docs](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle).\n",
    "\n",
    "**A note about grading:**\n",
    "\n",
    "**When you submit this assignment for grading these same values for these globals will be used so make sure that all your code works well with these values. After submitting and passing this assignment, you are encouraged to come back here and play with these parameters to see the impact they have in the classification process. Since this next cell is frozen, you will need to copy the contents into a new cell and run it to overwrite the values for these globals.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "SPLIT_TIME = 1100\n",
    "WINDOW_SIZE = 20\n",
    "BATCH_SIZE = 32\n",
    "SHUFFLE_BUFFER_SIZE = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, put everything together and create the times series you will use for this assignment. You will save them in the global variables `TIME` and `SERIES`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# Create the time series\n",
    "TIME, SERIES = generate_time_series()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Plot the generated series\n",
    "plt.figure(figsize=(10, 6))\n",
    "plot_series(TIME, SERIES)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Processing the data\n",
    "\n",
    "Since you already coded the `train_val_split` and `windowed_dataset` functions during past week's assignments, this time they are provided for you. Notice that in `windowed_dataset` an extra step is added which expands the series to have an extra dimension. This is done because you will be working with RNN-like layers which expect the dimensionality of its inputs to be 3 (including the batch dimension). In the previous weeks you used simple Dense layers which don't have this requirement.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def train_val_split(time, series):\n",
    "    \"\"\" Splits time series into train and validation sets\"\"\"\n",
    "    time_train = time[:SPLIT_TIME]\n",
    "    series_train = series[:SPLIT_TIME]\n",
    "    time_valid = time[SPLIT_TIME:]\n",
    "    series_valid = series[SPLIT_TIME:]\n",
    "\n",
    "    return time_train, series_train, time_valid, series_valid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "4sTTIOCbyShY",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def windowed_dataset(series, window_size):\n",
    "    \"\"\"Creates windowed dataset\"\"\"\n",
    "    series = tf.expand_dims(series, axis=-1)\n",
    "    dataset = tf.data.Dataset.from_tensor_slices(series)\n",
    "    dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
    "    dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
    "    dataset = dataset.shuffle(SHUFFLE_BUFFER_SIZE)\n",
    "    dataset = dataset.map(lambda window: (window[:-1], window[-1]))\n",
    "    dataset = dataset.batch(BATCH_SIZE).prefetch(1)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, run the cell below to call these two functions and generate your training dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "4sTTIOCbyShY",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Split the dataset\n",
    "time_train, series_train, time_valid, series_valid = train_val_split(TIME, SERIES)\n",
    "# Apply the transformation to the training set\n",
    "dataset = windowed_dataset(series_train, WINDOW_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining the model architecture\n",
    "\n",
    "### Exercise 1: create_uncompiled_model\n",
    "\n",
    "Now that you have a function that will process the data before it is fed into your neural network for training, it is time to define your layer architecture. \n",
    "\n",
    "In previous weeks or courses you defined your layers and compiled the model in the same function. However, here you will do thing a little bit different: you will first define the `create_uncompiled_model` function, which only determines your model's structure, and later on you will compile it. This way you can can reuse your model's layers for the learning rate adjusting and the actual training.\n",
    "\n",
    "Remember that, as you saw on the lectures, there are a couple of layers you will need to add. Firstly, since LSTM and RNN layers expect three dimensions for the input (`batch_size`, `window_size`, `series_dimensionality`), and you have just a univariate time series, you will need to account for this, which can be done via the `tf.keras.Input` (this is already provided for you). Also, it is a good practice to add a layer at the end to make the output values, which are between -1 and 1 for the tanh activation function, be of the same order as the actual values of the series. \n",
    "\n",
    "Hint:\n",
    "- You should use `SimpleRNN` or `Bidirectional(LSTM)` as intermediate layers.\n",
    "\n",
    "- The last layer of the network (before the last `Lambda`) should be a `Dense` layer.\n",
    "- Fill in the `Lambda` layer at the end of the network with the correct lambda function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: create_uncompiled_model\n",
    "def create_uncompiled_model():\n",
    "    \"\"\"Define uncompiled model\n",
    "\n",
    "    Returns:\n",
    "        tf.keras.Model: uncompiled model\n",
    "    \"\"\"\n",
    "    ### START CODE HERE ###\n",
    "    \n",
    "    model = tf.keras.models.Sequential([ \n",
    "        tf.keras.Input((WINDOW_SIZE, 1)), \n",
    "\t\t# Your layers go here\n",
    "        \n",
    "        tf.keras.layers.Lambda(None)\n",
    "    ]) \n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell allows you to check the number of total and trainable parameters of your model and prompts a warning in case these exceeds those of a reference solution, this serves the following 3 purposes listed in order of priority:\n",
    "\n",
    "- Helps you prevent crashing the kernel during training.\n",
    "\n",
    "- Helps you avoid longer-than-necessary training times.\n",
    "\n",
    "- Provides a reasonable estimate of the size of your model. In general you will usually prefer smaller models given that they accomplish their goal successfully.\n",
    "\n",
    "**Notice that this is just informative** and may be very well below the actual limit for size of the model necessary to crash the kernel. So even if you exceed this reference you are probably fine. However, **if the kernel crashes during training or it is taking a very long time and your model is larger than the reference, come back here and try to get the number of parameters closer to the reference.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define your uncompiled model\n",
    "uncompiled_model = create_uncompiled_model()\n",
    "\n",
    "# Check the parameter count against a reference solution\n",
    "unittests.parameter_count(uncompiled_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [],
   "source": [
    "example_batch = dataset.take(1)\n",
    "\n",
    "try:\n",
    "\tpredictions = uncompiled_model.predict(example_batch, verbose=False)\n",
    "except:\n",
    "\tprint(\"Your model is not compatible with the dataset you defined earlier. Check that the loss function and last layer are compatible with one another.\")\n",
    "else:\n",
    "\tprint(\"Your current architecture is compatible with the windowed dataset! :)\")\n",
    "\tprint(f\"predictions have shape: {predictions.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected output:**\n",
    "\n",
    "```\n",
    "Your current architecture is compatible with the windowed dataset! :)\n",
    "predictions have shape: (NUM_BATCHES, 1)\n",
    "```\n",
    "Where `NUM_BATCHES` is the number of batches you have set to your dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Test your code!\n",
    "unittests.test_create_uncompiled_model(create_uncompiled_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a last check, you can also print a summary of your model to see what the architecture looks like. This can be useful to get a sense of how big your model is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "uncompiled_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adjusting the learning rate - (Optional Exercise)\n",
    "\n",
    "As you saw in the lectures you can leverage Tensorflow's callbacks to dinamically vary the learning rate during training. This can be helpful to get a better sense of which learning rate better acommodates to the problem at hand.\n",
    "\n",
    "**Notice that this is only changing the learning rate during the training process to give you an idea of what a reasonable learning rate is and should not be confused with selecting the best learning rate, this is known as hyperparameter optimization and it is outside the scope of this course.**\n",
    "\n",
    "For the optimizers you can try out:\n",
    "- [`tf.keras.optimizers.Adam`](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam)\n",
    "- [`tf.keras.optimizers.SGD`](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD) with a momentum of 0.9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "def adjust_learning_rate(model):\n",
    "    \"\"\"Fit model using different learning rates\n",
    "\n",
    "    Args:\n",
    "        model (tf.keras.Model): uncompiled model\n",
    "\n",
    "    Returns:\n",
    "        tf.keras.callbacks.History: callback history\n",
    "    \"\"\"\n",
    "    \n",
    "    lr_schedule = tf.keras.callbacks.LearningRateScheduler(lambda epoch: 1e-6 * 10**(epoch / 20))\n",
    "    \n",
    "    ### START CODE HERE ###\n",
    "    \n",
    "    # Select your optimizer\n",
    "    optimizer = None\n",
    "    \n",
    "    # Compile the model passing in the appropriate loss\n",
    "    model.compile(loss=None,\n",
    "                  optimizer=optimizer, \n",
    "                  metrics=[\"mae\"]) \n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "    \n",
    "    history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])\n",
    "    \n",
    "    return history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Run the training with dynamic LR\n",
    "lr_history = adjust_learning_rate(uncompiled_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the achieved loss for each learning rate value, this way you can select an appropriate learning rate for your training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Plot the loss for every LR\n",
    "plt.semilogx(lr_history.history[\"learning_rate\"], lr_history.history[\"loss\"])\n",
    "plt.axis([1e-6, 1, 0, 30])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on this plot, which learning rate would you choose? You will get to use it on the next exercise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compiling the model\n",
    "\n",
    "### Exercise 2: create_model\n",
    "\n",
    "Now it is time to do the actual training that will be used to forecast the time series. For this complete the `create_model` function below.\n",
    "\n",
    "Notice that you are reusing the architecture you defined in the `create_uncompiled_model` earlier. Now you only need to compile this model using the appropriate loss, optimizer (and learning rate). If you completed the previous optional exercise, you should have a pretty good idea of which combinations might work better. \n",
    "\n",
    "Hint:\n",
    "- The training should be really quick so if you notice that each epoch is taking more than a few seconds, consider trying a different architecture.\n",
    "\n",
    "- If after the first epoch you get an output like this: `loss: nan - mae: nan` it is very likely that your network is suffering from exploding gradients. This is a common problem if you used `SGD` as optimizer and set a learning rate that is too high. **If you encounter this problem consider lowering the learning rate or using Adam with the default learning rate.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# GRADED FUNCTION: create_model\n",
    "def create_model():\n",
    "    \"\"\"Creates and compiles the model\n",
    "\n",
    "    Returns:\n",
    "        tf.keras.Model: compiled model\n",
    "    \"\"\"\n",
    "    model = create_uncompiled_model()\n",
    "\n",
    "    ### START CODE HERE ###\n",
    "\n",
    "    model.compile(loss=None,\n",
    "                  optimizer=None,\n",
    "                  metrics=[\"mae\"])  \n",
    "    \n",
    "    ### END CODE HERE ###\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create an instance of the model\n",
    "model = create_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Test your code!\n",
    "unittests.test_create_model(create_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now go ahead and train your model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Train it\n",
    "history = model.fit(dataset, epochs=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now go ahead and plot the training loss so you can monitor the learning process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Plot the training loss for each epoch\n",
    "\n",
    "loss = history.history['loss']\n",
    "\n",
    "epochs = range(len(loss))\n",
    "\n",
    "plt.plot(epochs, loss, 'r', label='Training loss')\n",
    "plt.title('Training loss')\n",
    "plt.legend(loc=0)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating the forecast\n",
    "\n",
    "Now it is time to evaluate the performance of the forecast. For this you can use the `compute_metrics` function that you coded in a previous assignment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def compute_metrics(true_series, forecast):\n",
    "    \"\"\"Computes MSE and MAE metrics for the forecast\"\"\"\n",
    "    mse = tf.keras.losses.MSE(true_series, forecast)\n",
    "    mae = tf.keras.losses.MAE(true_series, forecast)\n",
    "    return mse, mae"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you have trained the model that will perform the forecast, but you still need to compute the actual forecast. For this, you will use the `generate_forecast` function. This function, which is the same you used on previous assignments, generates the next value given a set of the previous `window_size` points for every point in the validation set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 388
    },
    "deletable": false,
    "editable": false,
    "id": "icGDaND7z0ne",
    "outputId": "278c6772-ccda-4a4b-9940-bbd9a69bfae6",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "def generate_forecast(model, series, window_size):\n",
    "    \"\"\"Generates a forecast using your trained model\"\"\"\n",
    "    forecast = []\n",
    "    for time in range(SPLIT_TIME, len(series)):\n",
    "        pred = model.predict(series[time-window_size:time][np.newaxis])\n",
    "        forecast.append(pred[0][0])\n",
    "    return forecast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, run the cells below to generate and plot the forecast series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 388
    },
    "deletable": false,
    "editable": false,
    "id": "icGDaND7z0ne",
    "outputId": "278c6772-ccda-4a4b-9940-bbd9a69bfae6",
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Save the forecast\n",
    "rnn_forecast = generate_forecast(model, SERIES, WINDOW_SIZE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 388
    },
    "deletable": false,
    "editable": false,
    "id": "icGDaND7z0ne",
    "outputId": "278c6772-ccda-4a4b-9940-bbd9a69bfae6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Plot your forecast\n",
    "plt.figure(figsize=(10, 6))\n",
    "\n",
    "plot_series(time_valid, series_valid)\n",
    "plot_series(time_valid, rnn_forecast)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected Output:**\n",
    "\n",
    "A series similar to this one:\n",
    "\n",
    "<div>\n",
    "<img src=\"images/expected.png\" width=\"650\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now use the `compute_metrics function` to find the MSE and MAE of your forecast. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "mse, mae = compute_metrics(series_valid, rnn_forecast)\n",
    "\n",
    "print(f\"mse: {mse:.2f}, mae: {mae:.2f} for forecast\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**You will be graded based on your model performance. To pass this assignment your forecast should achieve an MAE of 4.5 or less.**\n",
    "\n",
    "- If your forecast didn't achieve this threshold try re-training your model with a different architecture (you will need to re-run both `create_uncompiled_model` and `create_model` functions) or tweaking the optimizer's parameters.\n",
    "\n",
    "\n",
    "- If your forecast did achieve this threshold run the following cell to save your achieved MAE for the forecast, which will be used for grading. After doing so, submit your assignment for grading."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Save your mae in a pickle file\n",
    "with open('forecast_mae.pkl', 'wb') as f:\n",
    "    pickle.dump(mae.numpy(), f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Congratulations on finishing this week's assignment!**\n",
    "\n",
    "You have successfully implemented a neural network capable of forecasting time series leveraging Tensorflow's layers for sequence modelling such as `RNNs` and `LSTMs`! **This resulted in a forecast that matches (or even surpasses) the one from last week while training for half of the epochs.**\n",
    "\n",
    "**Keep it up!**"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "C4_W3_Assignment_Solution.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "grader_version": "1",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
